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Executive Summary 



This report has been prepared by the Educational Testing Service® (ETS) State and 
Federal Relations Office in response to recent inquiries about the potential effects 
of certain legislative proposals intended to improve teacher education. Several of 
these proposals call for specified minimum pass rates on state teacher licensing 
exams to establish institutional eligibility for certain federal funding. ETS is one 
of two testing organizations that offer assessments for state teacher licensure in the 
United States, and the only one that offers such assessments nationally. Thirty-one 
states plus the District of Columbia and the Virgin Islands will use The Praxis 
Series: Professional Assessments for Beginning Teachers® for teacher licensure, as 
of fall 1998. We estimate that over half of all licensure candidates tested annually 
take Praxis assessments. 

To help inform policy deliberations, we have conducted analyses on Praxis score 
data from the 1996-1997 testing year. This report examines the possible effects of 
proposals to eliminate the eligibility for certain federal funding of institutions of 
teacher education that fail to meet a specified pass rate on state teacher licensing 
exams for their graduates. Suggested required pass rates have ranged from 70% to 
90%. Some proposals have required that pass rates be based on those who passed 
the exam on the first attempt. We examine the potential effect of this requirement as 
well. 

Because of commitments to protect the confidentiality of data identified with 
examinees taking ETS tests, as well as states and institutions to whom the data are 
reported, ETS does not publicly disclose test scores or state or institutional pass 
rates on its assessments. We are, however, committed to sharing relevant data that 
pertain to pressing public policy issues. To this end, we are providing aggregated 
test information from the Praxis assessments for the 1996-1997 testing year such 
that particular states and institutions are not identifiable. 

This report briefly describes the Praxis 1 assessments of basic skills in reading, 
writing, and mathematics and the nearly 140 Praxis 11 subject assessments offered 
in more than 50 content areas. It then discusses issues relative to state qualifying 
scores and variations among them. Next, we describe both state and institutional 
pass rates, with particular attention directed to institutions with fewer than 10 
students taking a test. Data are provided to illustrate the effects of the proposed pass 
rate requirements and the first attempt provision. Our report concludes with some 
guidelines, concerns, and cautions that we think should be considered in the policy 
debates. 

Qualifying scores (also referred to as passing scores, passing standards, or cut 
scores) are set by states for each of the assessments they require for licensure. 

Since a qualifying score must be set for each test, states have many qualifying 
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scores. These scores vary considerably across tests and across states. The success 
rate of an institution depends in part on where this cut point has been set by the 
state. Recent state qualifying scores on six subject tests ranged from below the 25th 
percentile to above the 50th percentile, depending on the test, of the score 
distributions of all test-takers. (See Table 1.) 

Pass rates on Praxis exams indicate the percentage of examinees that met or 
exceeded the qualifying score for the particular assessment set by the state that 
requires the test. Such pass rates vary across states and institutions, depending on 
state qualifying scores and on the performance of the teacher candidates each year. 

A state or institution can have many pass rates, depending on the number of exams 
required for licensure. To inform policy deliberations on the important issue of 
new teachers' content knowledge in the subject they are about to teach, we selected 
six illustrative Praxis II subject assessments for analysis, including a test in 
elementary education, English, math, science, social studies, and a foreign 
language, as follows: 

(1) Elementary Education: Curriculum, Instruction, and Assessment 

(2) English Language, Literature, and Composition: Content Knowledge 

(3) Mathematics: Content Knowledge 

(4) Biology and General Science 

(5) Social Studies: Content Knowledge 

(6) Spanish: Content Knowledge 

State pass rates . Table 2 shows the pass rates anonymously on these tests for the 
states that require them. The pass rates vary considerably by state and by test, 
depending on each state's qualifying scores for each test, as well as on the 
performance of the teacher candidates trained within the state. Pass rates were 
lowest in mathematics and highest in elementary education. 

Institutional pass rates . We point out that pass rates based on a small group of students 
can be overly influenced by the performance of a single student . For example, if only six 
students are being tested, one additional student passing a test can raise the pass 
rate from below 70% to above 80%. Alternatively, one student failing instead of 
passing could lower the pass rate from above 80% to below 70%. Fluctuations 
associated with pass rates based on a few students could result in institutions being 
given an unfair advantage or disadvantage with respect to meeting the pass rate. 
Our analysis shows tha t over three-fourths of the institutions preparing students in the 
six content areas were testing fewer than 10 students in 1996-1997 . (See Table 3.) 

Table 4 shows the effect on teacher preparation institutions of a 70%, a 75%, or an 
80% pass rate requirement for funding eligibility. It presents data only for 
institutions with 10 or more students taking one of the six illustrative content 
knowledge tests. With the exception of those preparing students for the 
Mathematics: Content Knowledge test, relatively few institutions would be 
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ineligible based on their pass rates at the 70% pass rate requirement. On the math 
test, however, nearly 80% of the institutions (23 out of 29) would be ineligible at 
that requirement. The impact of increasing the required pass rate also varies 
according to the subject test. For four of the six tests, relatively few institutions 
would become ineligible if the pass rate increased from 70% to 75% or from 75% to 
80%. For English and for social studies, however, the number of ineligible 
institutions would more than double if the pass rate requirement were to increase 
from 70% to 75% and more than triple if the pass rate requirement changed from 
70% to 80%. At the 80% pass rate requirement, twenty-six (90 percent) of the 
institutions preparing students for the math test and twenty-four (over 50 percent) 
preparing for the social studies exam, would be ineligible. For the four other 
subject tests, from four to fourteen (13 percent to 44 percent) of the institutions — 
depending on the exam— would become ineligible given an 80% pass rate 
requirement. 

“Pass on first attempt," a requirement for computing pass rates in some early 
legislative proposals, has generated considerable discussion. Our results from a 
limited analysis show small increases in pass rates by using data based on 
students' opportunity to re-take a test in the same testing year. Gains from 
additional attempts on Praxis I basic skills tests pass rates were minimal, ranging 
from two to three percentage points. Small, but slightly larger gains were shown 
for the six illustrative Praxis II subject tests. The largest pass rate increase, of eight 
percentage points from first to last attempt, was on the Mathematics: Content 
Knowledge test. (See Table 5.) We note, however, that the findings of the "first 
attempt" analyses shown on this table should be viewed with caution, as the data 
were drawn from just one testing year. If students' attempts before and after that 
year had been included, the gains from first to last attempt would have been 
larger, as discussed on page 13. 

Conclusions. ETS recommends that policy-makers exercise caution in requiring 
minimum pass rates for institutions. The use of pass rates on teacher licensing 
exams as the sole criterion for institutional eligibility for funding would be 
inconsistent with good professional practice. Our guidelines for the use of Praxis 
scores, summarized in this document, point out that institutional efficacy is 
indicated by many factors, only one of which is the performance of graduates on 
licensing exams. We further recommend that tests be validated for the specific 
purposes for which they are used, including the evaluation of teacher preparation 
programs. 

The use of pass rates to determine funding eligibility could result in inequities to 
institutions producing only a few teacher education graduates annually. Over 
three-quarters of the institutions studied for this analysis were testing fewer than 
10 students in a subject. Pass rates based on such small numbers are unstable, as 
they are overly influenced by the performance of a single student. We thus 
encourage policy-makers to take this important concern into account as legislation 
is considered. 
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States use varying licensing tests with varying levels of difficulty and varying 
qualifying scores, and nine states do not test for teacher licensure. The states that 
use ETS assessments choose from nearly 140 Praxis subject tests. Thus, there is no 
comparability from state to state unless states use the same tests with the same 
qualifying scores. Some examinees seek licensure in a state other than the one in 
which they received their teacher preparation, and the "receiving" state is likely to 
have qualifying scores that are different from those of the "sending" state. Some 
states require that prospective teachers pass several tests in order to be licensed. 

A single pass rate requirement, therefore, would not be equitably applied across 
states and would not represent a nationally uniform standard. The already 
federally sanctioned professional accreditation process does offer such a standard. 
We respectfully suggest that policy-makers explore the philosophy and intent of 
the NCATE 2000 initiative of the National Council for Accreditation of Teacher 
Education, which is designed to link program accreditation with licensing 
outcomes . 

This paper is presented to inform policy debate and to generate constructive 
discussion, and we welcome comments and questions from colleagues. For 
questions regarding the findings of the study, please contact: Penelope Engel of 
the ETS State and Federal Relations Office, Washington, D.C. at 202/ 659-8043 (or 
pengel@ets.org), or Ines Bosworth of the Praxis Program in the ETS Midwestern 
Office, Evanston, Illinois, at 847/869-7700 (or ibosworth@ets.org). 
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I. Introduction 



This background report was prepared by ETS in response to requests from 
Congressional staff, education association and interest group representatives, as 
well as the Congressional Research Service, for information on pass rates on the 
Praxis teacher licensing exams. As the reauthorization of the Higher Education Act 
has proceeded through the 105th Congress, significant attention has focused on the 
current Title V (new Title 11), concerning the preparation and development of 
teachers. More than 10 bills or legislative proposals have been presented for 
consideration in recent months, each suggesting incentives to improve teacher 
education. One category of incentives offered in several proposals is the 
requirement that, in order to be eligible for federal funds for teacher preparation 
purposes, teacher education programs at collegiate institutions must achieve 
minimum specified pass rates on their state's teacher licensing exams. 

As this paper goes to press, the House has already passed a bill to reauthorize the 
Higher Education Act, and the Senate is expected to vote on its bill shortly. 
Provisions concerning improvement of teacher education programs are contained 
in both bills, and conferees will have the responsibility of resolving the 
differences. 

A House provision requires that in order to be eligible for funding for 
"exemplary institution" partnerships, a state must show that at least 80 percent of 
the teacher education graduates of such institutions pass assessments of subject 
matter knowledge in the content areas(s) in which they will teach. A 70% pass rate 
is required of all teacher education programs in the state for future receipt of 
teacher preparation funds under the act. A related provision requires states to 
report to the Secretary of Education those programs that receive federal funds and 
have less than a 70% pass rate on any of the state's initial teacher licensing 
assessments. 

A proposal being considered in the Senate, now undergoing revision, also includes 
a pass rate requirement. This proposal requires that, in order to receive federal 
funds for teacher preparation purposes, a teacher education institution must either 
meet professional accreditation standards or show that 75 percent of its graduates 
pass their license exam. Exams must include an assessment of teachers' content 
area knowledge in the area(s) in which they plan to teach. Earlier language 
requiring students to pass their exam on their first attempt, has now been dropped. 

ETS offers examinations nationally to states for teacher licensure through its NTE 
(National Teacher Examinations) testing program and its successor. The Praxis 
Series: Professional Assessments for Beginning Teachers®. Not surprisingly, 
questions as to differences in state qualifying scores and the effects of various 
proposed pass rates on institutions of teacher education have been directed to ETS. 
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The closer these issues are examined, the more their complexities become evident, 
as explained below. Although ETS desires to provide data to inform public policy 
debates, this desire is superseded by our confidentiality commitments to test-takers 
as well as to Praxis clients. Thus, this document provides aggregate information 
without associating data with any particular student, state, or institution. 

ETS is not endorsing a particular legislative proposal at this time. Our purpose is to 
shed light on the complexities surrounding the use of pass rates and to point out 
caveats and cautions that should be kept in mind. Our intent is to assist the reader 
in understanding the numerous variations in teacher licensing requirements as well 
as the potential effects of specified pass rates on institutions in the aggregate. 

ETS has developed and administered the National Teacher Examinations since 1948. 
During that period, NTE became widely used across the nation, both for purposes 
of state licensing and for entry into institutional teacher preparation programs. 

Over time, many subject area and specialized tests were developed and added to 
the NTE complement. In 1987, ETS initiated a very large research and development 
effort to create a new series of program entry and licensing exams. The Praxis 
Series: Professional Assessments for Beginning Teachers®, to ultimately replace the 
NTE. Currently, some states are using Praxis tests and others are in a transition 
period, as they change from NTE to Praxis. (Hereafter in this report, "Praxis" will 
be used to refer to both NTE and Praxis exams.) 

ETS is one of two testing organizations that offer teacher assessments for state 
licensing in the United States. As of fall 1998, 31 states plus the District of Columbia 
and the Virgin Islands will use one or more Praxis exams for state teacher licensure. 
We estimate that over half of all candidates tested annually for licensure take Praxis 
assessments. National Evaluation Systems, of Amherst, Mass., provides licensure 
testing services to nine states and shares licensure testing with ETS in one state. An 
additional nine states (Alabama, Idaho, Iowa, North Dakota, South Dakota, Utah, 
Vermont, Washington, and Wyoming) do not test for teacher licensing at this time. 

This paper first provides a description of the Praxis exams, followed by a discussion 
of scores on the tests, how ETS reports them, and the variation in states' qualifying 
scores on these exams. We then provide information about pass rates and the 
potential effects of those specified in current legislative proposals. Next, we discuss 
the matter of pass rates based on test-takers' "first attempt," compared to pass rates 
based on any number of attempts within a testing year. We conclude with 
guidelines for the use of Praxis scores and a summary of concerns and cautions. 
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II. The Praxis Assessments 



Praxis assessments, along with other criteria, are used for three primary purposes: 

1. by state agencies, for licensing teachers and other school professionals 

2. by higher education governing agencies and educational institutions, for 
assessing the readiness of applicants seeking entry into a teacher education 
program, typically in their junior year 

3. by state agencies and educational institutions, to evaluate the effectiveness of 
undergraduate teacher training programs. 

The Praxis Series™ includes: 

Praxis I: Academic Skills Assessments. These Reading, Math, and Writing 

assessments test the basic skills that have been identified as needed by 
beginning teachers. They are taken by students whose institutions 
require Praxis I scores for entry into their programs. In addition, they 
are taken by students for licensure, as required by 19 states plus the 
District of Columbia and the Virgin Islands. The tests are available in 
both pencil-and-paper and computer-based formats.^ 

Praxis II: Subject Assessments. The nearly 140 tests in more than 50 areas 

measure knowledge of specific academic subjects or fields and are 
designed to evaluate the knowledge and skills prospective teachers 
acquire in their academic preparation. These tests are taken for 
licensure to teach in a particular content area. Praxis II tests also 
include the three Principles of Learning and Teaching Tests (Grades K- 
6, Grades 5-9, and Grades 7-12), which assess a preservice, begirming, 
or provisional teacher's knowledge of a variety of job-related criteria. 
Praxis II tests are required by 27 states plus the District of Columbia. 

Praxis III: Classroom Performance Assessments. These assessments combine in- 

class observation of teaching performance with written documentation 
and pre- and post-observation interviews. Trained assessors 
contribute to state licensing decisions through the use of a common 
evaluation framework. Praxis III is currently being pilot tested in the 
state of Ohio. 



’ The Praxis I pencil-and-paper basic skills tests are known as the Pre-Professional Skills Tests 
(PPST). The Praxis I computer-based basic skills tests are referred to as "CBT." 
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III. Prsixis Scores 



Praxis scores are provided to three entities: 

* the responsible state agency 

* the institution the candidate is attending 

* the individual examinee 

Because of commitments to protect the confidentiality of data identified with examinees 
taking ETS tests, as well as states and institutions to whom the scores are reported, ETS does 
not provide Praxis scores or pass rates to entities other than those listed above. Interested 
parties may contact the appropriate state department of education to ascertain the public 
availability of this information. 

In accordance with sound professional practice, as expressed in the standards of the 
profession, ETS standards, and the specific guidelines of the Praxis program (See 
Section IV, page 15), Praxis scores may not be used: 

* as the sole criterion for any decision 

* without appropriate validation 

* without appropriate standard-setting studies 

* to rank candidates or programs for presumed quality 

* by school districts to make employment decisions relative to licensed 
teachers 



State Qualifying Scores 

Praxis user states set qualifying scores (also referred to as passing scores, passing 
standards, or cut scores) on each of the assessments they require for licensure. 

These qualifying scores vary considerably across tests, depending on each test's 
level of difficulty, and across states. For example, in a recent year, the state of Ohio 
required 31 different Praxis assessments, and Indiana required 36 assessments, each 
with its own state-established qualifying score. Some examinees seek licensure in 
a state other than the one in which they received their teacher preparation. Such a 
"receiving" state is likely to have different qualifying scores for teacher licensure 
on one or more of its tests from those of the "sending" state. 

The qualifying scores set by states are public information and are published bi- 
armually by ETS.^ For example, qualifying scores on the Praxis 1 Pre-Professional 



* See the attached publication, "Understanding Your Praxis Scores, 1997-98 Spring Edition." 
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Skills Tests, set by the 19 states plus the District of Columbia and the Virgin Islands 
that use them for licensing, range from 169 to 178 on the Mathematics and Reading 
tests, and from 168 to 176 on the Writing test. Qualifying scores on one or more 
Praxis II Subject Assessments have been set by 27 states and the District of Columbia 
for licensing purposes. These scores on the nearly 140 subject assessments in more 
than 50 fields also vary widely across subjects and states. 

Table 1 presents the range of qualifying scores across participating states for six 
illustrative Praxis II content area tests. The state qualifying scores range from below 
the 25th percentile to above the 50th (but below the 75th) percentile of the score 
distributions of all test-takers, depending on the test.^ Using Elementary 
Education— the first test listed — as an example, the table shows that at least one 
state has set its qualifying score on that test at 143, and that at least one other state 
has set its qualifying score at 164, and that the qualifying scores for the remaining 
states fall within that 143-164 range. The table also shows that the qualifying scores 
of all states for this test fall below the 25th percentile of the distribution of all test- 
takers' scores. 4 

It is apparent that in the absence of uniform qualifying scores across states, the 
impact of a requirement for a specified pass rate on a state's teacher licensing exam 
would differ across states. 

ETS does not set qualifying scores but is available to provide consultation to states 
regarding the process. Each state, agency, or institution considering establishing a 
qualifying score on one of the Praxis assessments must conduct a standard-setting 
study using a technically and legally acceptable method. Evidence that supports 
the use of the qualifying score is gathered to assure that the score is consistent with 
a reasonable judgment of the minimum required academic knowledge and skills. 
Each state is expected to periodically review and update this process. 



® "Percentile" refers to a point in the score distribution below which the stated percentage of all 
scores fall. A test-taker scoring at the 25th percentile has scored higher than 25 percent of the 
students; a test-taker scoring at the 75th percentile has scored higher than 75 percent of the students. 

"'However, for a given state and a given test, the pass rate among that state's students is likely to be 
different from the pass rate that would result from applying the state's passing score to all test- 
takers. 



Table 1 

Score Distribution Percentiles and Range of State Qualifying Scores 
on Selected Teacher Licensure Subject Tests 
1996-1997 Testing Year 



Praxis II Subject 
Assessments 
(six illustrative tests) 


Score Distribution Percentiles 
(among all test-takers) 


Range of 
State 

Qualifying 

Scores 




25th 

Percentile 


50th 

Percentile 


75th 

Percentile 




Elementary Education: 
Curriculum, Instruction, and 
Assessment 


167 


179 


188 


143-164 


English Language, 
Literature, and 
Composition: 
Content Knowledge 


165 


177 


187 


138-172 


Mathematics: Content 
Knowledge 


120 


137 


153 


124-147 


Biology and General Science 


580 


630 


690 


540-590 


Social Studies: Content 
Knowledge 


156 


168 


178 


134-162 


Spanish: Content 
Knowledge 


160 


176 


188 


143-171 



Note: Score distribution percentile data refer to the group of all examinees who took these tests 
between October 1 , 1996 and September 30, 1997. The Biology and General Science test is scored on 
the NTE scale of 250-990; the others are scored on the Praxis scale of 100 to 200. The range of 
qualifying scores is as of August 1997. 



Praxis Pass Rates 

Pass rates on Praxis exams indicate the percentage of examinees who met or 
exceeded the qualifying score for the particular assessment set by the state that 
requires the test. Such pass rates vary across states depending on the qualifying 
scores and on performance of the teacher candidates each year. Rates for each 
particular Praxis assessment can be calculated for each state and for each institution 
attended by the teacher candidates. Thus, any state or institution can have a 
multitude of pass rates, depending on the number of Praxis exams required of its 
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teacher candidates for licensure. It is difficult to compare pass rates across states fairly 
because of differences in state Qualifying scores . 

State Pass Rates, In an attempt to address the important policy issue of new 
teachers' content knowledge of the subject they are about to teach, ETS prepared 
Table 2, “Pass Rates for Individual States (Not Identified) on Selected Subject Tests for 
Teacher Licensure. " Although the names of the states that require each of the six 
tests are public information, they are listed by number only in this table so that 
their identity and their pass rates are anonymous. The order in which they are listed 
has been randomized separately for each test . For example, the state listed as #3 in one 
column is not necessarily the same state listed as #3 in any other column. This 
table reports state pass rates on Praxis II subject tests in the six illustrative areas. 
We have reported pass rates only for those states that tested at least 20 candidates 
in the 1996-1997 testing year on a particular test. Each of these six tests is a 
requirement for licensure in at least eight states. Not all the states require all six 
tests. 

The reader can see that the pass rates vary considerably by state and by test. As 
explained above, this variation is affected by the particular qualifying score 
established by each state for each test. We can see, for the illustrative tests in the 
table, that pass rates are lowest in Mathematics: Content Knowledge and highest 
in Elementary Education. The number of test-takers for each of the tests in the 
1996-1997 testing year is shown on Table 5. 
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Institutional Pass Rates. An institutional pass rate is the percentage of candidates 
at each teacher preparation institution who meet or exceed the state's qualifying 
score. Institutional pass rates depend on the qualifying scores set by each state and 
on the performance of the teacher candidates each year. 

At the institutional level it is common to find very small programs graduating 
from five to 10 prospective teachers in any given subject area. ETS, in fact, does not 
report pass rates based on fewer than 10 test-takers. One of the reasons for this 
policy is that a pass rate based on a small group of students can be overly influenced by the 
performance of a single student For example, if 10 students from an institution were 
being tested for licensure on a particular test, the change of one person from pass 
to fail could lower the institution's pass rate from 80% to 70%. If only six persons 
are being tested, one additional student passing a test can raise the pass rate from 
below 70% to above 80%. If only two students from an institution are taking a 
particular test in one year (which is quite common), there are only three possible 
pass rates: 0%, 50%, and 100%, and the change of one student from pass to fail or 
from fail to pass will alter the pass rate by 50 points. 

Fluctuations are associated with pass rates calculated on a small number of test- 
takers, and the smaller the number, the greater the fluctuations. Such fluctuations in 
pass rates could cause unfairness to institutions testing only a few teacher education 
graduates per year. Their effect could result in institutions being given an advantage 
or a disadvantage with respect to meeting the pass rate. The institutions affected 
could be small or large but are those from which only a few students are tested per 
year on a licensing exam. Institutions that prepare larger numbers of candidates 
for a particular exam would be far less likely to have their funding depend on the 
performance of a single student. 

Table 3, which follows, shows, for the six illustrative tests, the number and 
percentage of institutions with fewer than 10 students taking the test. The 
percentages of such institutions range from 68% to 90%, depending on the 
particular test. It is apparent that, on average, over three- fourths of the institutions 
preparing students to teach in these six subject areas were testing fewer than 10 students.^ 



® An institution preparing students for two or more of these six tests is counted once for each test 
taken by any of its students. 
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Table 3 

Institutions with Fewer than 10 Students 
Taking Selected Praxis Subject Assessments 
1996-1997 Testing Year 



Praxis II Subject Area Assessments 
[six illustrative tests] 


Number of 
institutions 
with 
students 
tested in 
these 
subjects 


Institutions 
with fewer than 
10 students 
taking the test 






Number 


Percent 


Elementary Education: Curriculum, Instruction, 
and Assessment 


95 


65 


68% 


English Language, Literature, and Composition: 
Content Knowledge 


251 


175 


70% 


Mathematics: Content Knowledge 


155 


126 


81% 


Biology and General Science 


158 


131 


83% 


Social Studies: Content Knowledge 


181 


134 


74% 


Spanish: Content Knowledge 


88 


79 


90% 


TOTAL: 


928 


710 


77% 



Note: An institution preparing students for two or more of these six tests is counted once for each test 
taken by any of its students. Only those institutions located in states that have established a single 
qualifying score for the test are included. Each tesNtaker is counted only once, regardless of the number 
of attempts. Institutional affiliation is based on the test-taker's listing of "College where you received 
training relevant to the test" and may include test-takers who are not current students or recent 
graduates. 



Table 4 reports on teacher preparation institutions with 10 or more students who took 
one of the six illustrative tests in the 1996-1997 testing year. Among these institutions, 
with the exception of those preparing students for the Mathematics: Content 
Knowledge test, relatively few (seven or fewer on any exam) would have been 
ineligible for funding based on their pass rates given a 70% pass rate requirement. On 
the math test, however, nearly 80% of the institutions (23 out of 29) would have been 
ineligible at the 70% requirement. 

The impact on institutions of increasing the required pass rate also varies according to 
the subject test. For four of the six tests, relatively few institutions would become 
ineligible if the pass rate requirement increased from 70% to 75% or from 75% to 80%. 
For English and for social studies, however, the number of ineligible institutions 
would more than double if the pass rate requirement were to increase from 70% to 
75% and would more than triple if the pass rate requirement changed from 70% to 
80%. At the 80% pass rate requirement, twenty-six (over 90 percent) of the institutions 
preparing students for the math test and twenty-four (over 50 percent) preparing 
students for the social studies exam, would be ineligible. For the four other subject 
tests, from four to fourteen (13 percent to 44 percent) of the institutions — depending 
on the exam— would have become ineligible given an 80% pass rate requirement. 

It should be noted that in some cases the same institution may be represented in the 
statistics for two or more of the tests summarized in Table 4. 
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Table 4 

Institutions Not Achieving Specified Pass Rates 

on Selected Praxis Subject Assessments 
Includes only institutions with 10 or more students taking the test 

1996-1997 Testing Year 



Praxis II Subject Area 
Assessments 




At least 
a 70% 
Pass 
Rate 


At least 
a 75% 
Pass 
Rate 


At least 
an 80% 
Pass 
Rate 


[six illustrative tests] 


Total 

number 

of 

institu- 

tions 


Number of 
institu- 
tions not 
meeting 
pass rate 


Number of 
institu- 
tions not 
meeting 
pass rate 


Number of 
institu- 
tions not 
meeting 
pass rate 


Elementary Education: 
Curriculum, Instruction, and 
Assessment 


30 


2 


3 


4 


English Language, Literature and 
Composition: 

Content Knowledge 


76 


4 


9 


14 


Mathematics: 
Content Knowledge 


29 


23 


25 


26 


Biology and 
General Science 


27 


5 


5 


8 


Social Studies: 
Content Knowledge 


47 


7 


15 


24 


Spanish: 

Content Knowledge 


9 


3 


3 


4 



Note: Scores earned by students attending each institution are compared with the qualifying score of the 
state in which the institution is located. Only those institutions located in states that have established a 
single qualifying score for the test are included. Each institution's pass rate refers to the group of 
students attending that institution that took the test in Testing Year 1996-1997. Each test-taker is counted 
only once, regardless of the number of attempts. Pass/fail status is based on the test-taker's last attempt 
in Testing Year 1996-1997. Institutional affiliation is based on the test-taker's listing of "College where 
you received training relevant to the test" and may include test-takers who are not current students or 
recent graduates. 
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Pass on First Attempt 



Some legislative proposals have required that institutional pass rates be calculated , 
on the basis of those students who passed a particular licensing exam on the first 
attempt The appropriateness of this language has generated considerable debate. 
Some students wish to "practice" by taking a test well in advance of when they 
need their license. Further, it is known that licensure candidates in other 
professions commonly take their licensure exams more than once. How those 
professions calculate their pass rates (on first attempt versus on any attempt) was 
not examined by this study. It is important to keep in mind that the first attempt 
proposals concern how a pass rate would be calculated for purposes of federal teacher 
education funding. They do not require states to restrict the number of tries 
allowed a candidate taking an exam for teacher licensure or for hiring. 

The institutional pass rates that ETS routinely calculates are based on each student's 
most recent score within the testing year. Thus, if a student took a test and failed it 
one or more times— but ultimately passed it during that year — that student would 
be counted as having passed the test in terms of the institution's pass rate. It is also 
possible to calculate pass rates based on students' first attempt in the testing year. 

The question as to the effect on pass rates of including scores of those allowed to 
re-take an exam is addressed by Table 5, “Percent of Praxis Test-Takers Passing 
Licensing Exams on First Attempt and on Last Attempt in 1996-1997 Testing Year" The 
data in general show small increases in pass rates based on test-takers' scores on 
repeat attempts in the same testing year. From analyses of Praxis I basic skills tests 
scores, we found minimal gains (from two to three percentage points) when the 
students counted as passing included those passing after more than one attempt. 
(These increases, however, represented from 766 to 1172 test-takers passing the test 
after failing on first attempt.) Small, but slightly larger gains from re-take 
opportunities were found for students taking the six Praxis II subject tests. The 
largest pass rate increase, of 8.3 percentage points from first to last attempt, was on 
the Mathematics: Content Knowledge test, representing 76 additional examinees. 

These data are limited, however, in that they cover only one testing year (1996- 
1997). If students' attempts before and after that year had been included, the gains 
from first to last attempt would have been larger. In this analysis, the number of 
test-takers who passed on their first attempt in 1996-1997 includes some who took 
the test before that year and failed. Their inclusion raises the percent of those 
passing on first attempt. Further, the number of test-takers who passed on their 
last attempt in 1996-1997 does not include those who failed, re-took the test after 
that year, and passed. Their exclusion lowers the percent of those passing on last 
attempt. Both of these effects reduce the difference between the percentage passing 
on first and on last attempt. 
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Table 5 

Percent of Praxis Test-Takers Passing Licensing Exams 
on First Attempt and on Last Attempt in 1996-1997 Testing Year 
Praxis I Academic Skills Assessments and six illustrative Praxis II Subject Tests 



CC 

c\? 



Increase from first to 
last attempt in 
1996-1997 


Percent 




2.0% 


3.0% 


2.4% 




2.1% 


2.8% 


8.3% 


3.9% 


4.4% 


3.8% 


Number of 
test-takers 




766 


1172 


965 




30 


62 


76 


30 


63 


15 


Percent 
passing on last 
attempt in 
1996-1997 






83% 


80% 


75% 




93% 


90% 


57% 


79% 


80% 


81% 


Percent 
passing on 
first attempt 
in 1996-1997 






81% 


77% 


73% 




91% 


Lv 

00 


49% 


75% 


76% 


77% 


Number of 
test-takers in 
1996-1997 






37,837 


39,578 


39,693 




1,415 


2,231 


916 


774 


1,437 


391 


Selected Praxis Assessments 




Academic Skills Assessments:* 


Reading (PPST and CBT) 


Writing (PPST and CBT) 


Mathematics (PPST and CBT) 


Selected Subject Assessments: 


Elementary Ed.: Curriculum, 
Instruction, and Assessment 


English Language, Literature and 
Composition: Content Knowledge 


Mathematics: Content Knowledge 


Biology and General Science 


Social Studies: Content Knowledge 


Spanish: Content Knowledge 
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IV. Guidelines for the Use of Praxis Scores 



Data on the performance of examinees graduating from particular teacher 
education programs may provide information relevant to the quality of such 
programs. However, ETS urges that limitations be imposed on conclusions 
reached solely on the basis of Praxis pass rates. We distinguish between the use of 
Praxis performance data to conclude that individual students have attained an 
adequate level of preparedness for teaching (a licensing use) and the use of scores 
to determine whether an educational program has accomplished its educational 
mission (an evaluation use). The latter use requires significantly more 
information than that represented by student performance on Praxis tests. 

Sound professional practice requires that the Praxis assessments be validated for 
the specific purposes for which they are being used.^ ETS offers the following 
further guidelines to ensure appropriate use of Praxis scores when applied to 
teacher education program evaluation. We recommend that the evaluating state 
agency: 

a. use multiple criteria that address the program's own mission and goals 

b. collect pre- and post-assessment data on the level of student attainment in 
order to gauge progress during the program 

c. set attainment goals for program graduates who are part of the evaluation 
collaboratively with the institution 

d. establish with each institution a timeline for implementation of the 
evaluation and publish information on use of test data in the process 

e. demonstrate that the content of subject assessments used to judge program 
quality reflects the material taught, assuring the student an opportunity-to- 
learn 

f. assure that data included in the evaluation are from only those students who 
received a substantial portion of their preparation in the program being 
evaluated 



* See American Educational Research Association, American Psychological Association, National 
Council on Measurement in Education. (1985) Standards for educational and psychological testing. 
Washington, DC: American Psychological Association, p. 67; Educational Testing Service. (1987) 
ETS standards for quality and fairness. Princeton, NJ: Author, p. 21; and Educational Testing Service, 
(in press). Guidelines for proper use of the Praxis Series and related assessments. Princeton, NJ: Author. 
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V. Conclusions 



ETS recommends that policy-makers exercise caution in requiring minimum pass 
rates for institutions. The use of pass rates on teacher licensing exams as the sole 
criterion for institutional eligibility for funding would be inconsistent with good 
professional practice. Our guidelines for the use of Praxis scores, summarized in 
this document, point out that institutional efficacy is indicated by many factors, 
only one of which is the performance of graduates on licensing exams. We further 
recommend that tests be validated for the specific purposes for which they are 
used, including the evaluation of teacher preparation programs. 

The use of pass rates to determine funding eligibility could result in inequities to 
institutions producing only a few teacher education graduates annually. Over 
three-quarters of the institutions studied for this analysis were testing fewer than 
10 students in a subject. Pass rates based on such small numbers are unstable, as 
they are overly influenced by the performance of a single student. We thus 
encourage policy-makers to take this important concern into account as legislation 
is considered. 

States use varying licensing tests with varying levels of difficulty and varying 
qualifying scores, and nine states do not test for teacher licensure. The states that 
use ETS assessments choose from nearly 140 Praxis subject tests. Thus, there is no 
comparability from state to state unless states use the same tests with the same 
qualifying scores. Some examinees seek licensure in a state other than the one in 
which they received their teacher preparation, and the "receiving" state is likely to 
have qualifying scores that are different from those of the "sending" state. Some 
states require that prospective teachers pass several tests in order to be licensed. 

A single pass rate requirement, therefore, would not be equitably applied across 
states and would not represent a nationally uniform standard. The already 
federally sanctioned professional accreditation process does offer such a standard. 
We respectfully suggest that policy-makers explore the philosophy and intent of 
the NCATE 2000 initiative of the National Council for Accreditation of Teacher 
Education, which is designed to link program accreditation with licensing 
outcomes. 
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